Signals Of Different Modalities Research Articles

Coevolution of social and communicative complexity in lemurs.

The endemic lemurs of Madagascar (Lemuriformes: Primates) exhibit great social and communicative diversity. Given their independent evolutionary history, lemurs provide an excellent opportunity to identify fundamental principles in the coevolution of social and communicative traits. We conducted comparative phylogenetic analyses to examine patterns of interspecific variation among measures of social complexity and repertoire sizes in the vocal, olfactory and visual modality, while controlling for environmental factors such as habitat and number of sympatric species. We also examined potential trade-offs in signal evolution as well as coevolution between body mass or brain size and communicative complexity. Repertoire sizes in the vocal, olfactory and visual modality correlated positively with group size, but not with environmental factors. Evolutionary changes in social complexity presumably antedated corresponding changes in communicative complexity. There was no trade-off in the evolution of signals in different modalities and neither body mass nor brain size correlated with any repertoire size. Hence, communicative complexity coevolved with social complexity across different modalities, possibly to service social relationships flexibly and effectively in pair- and group-living species. Our analyses shed light on the requirements and adaptive possibilities in the coevolution of core elements of social organization and social structure in a basal primate lineage. This article is part of the theme issue 'Cognition, communication and social bonds in primates'.

How signals interact in multimodal displays: Insights from a robotic frog.

Caldart, M. V., M. B. dos Santos & G. Machado (2021). Function of a multimodal signal: a multiple hypothesis test using a robot frog Journal of Animal Ecology, https://doi.org/10.1111/1365-2656.13620. Animals can communicate using signals perceived by different sensory systems, and many combine multiple sensory modalities in their display repertoires. Why these multimodal displays evolve and how they function to transmit information between individuals are crucial questions in behavioural and evolutionary research. Most empirical studies addressing these questions assume, even if implicitly, that signals of different modalities have independent effects on receiver responses. Nonetheless, the potential for interactions between signals as an explanation for the function of multimodal displays has been recognized for over two decades. Caldart et al. (2021) use a robotic frog and a receiver-based approach to test four alternative hypotheses for the function of multimodal (acoustic+visual) displays in the stream-dwelling frog Crossodactylus schmidti. Their results lend support to an inter-signal interaction mechanism, whereby inclusion of visual signals modifies the context in which an acoustic display is interpreted. In contrast, the results in Caldart et al. (2021) are less consistent with the hypotheses that emphasize the quality-related information encoded in different signal modalities and a hypothesis that focuses on signal transmission across heterogeneous environments. These results showcase how experimental manipulation of different signal modalities and tests of multiple alternative hypotheses are key to clarifying the function of multimodal displays.

Understanding and Synergy: A Single Concept at Different Levels of Analysis?

Biological systems differ from the inanimate world in their behaviors ranging from simple movements to coordinated purposeful actions by large groups of muscles, to perception of the world based on signals of different modalities, to cognitive acts, and to the role of self-imposed constraints such as laws of ethics. Respectively, depending on the behavior of interest, studies of biological objects based on laws of nature (physics) have to deal with different salient sets of variables and parameters. Understanding is a high-level concept, and its analysis has been linked to other high-level concepts such as “mental model” and “meaning”. Attempts to analyze understanding based on laws of nature are an example of the top-down approach. Studies of the neural control of movements represent an opposite, bottom-up approach, which starts at the interface with classical physics of the inanimate world and operates with traditional concepts such as forces, coordinates, etc. There are common features shared by the two approaches. In particular, both assume organizations of large groups of elements into task-specific groups, which can be described with only a handful of salient variables. Both assume optimality criteria that allow the emergence of families of solutions to typical tasks. Both assume predictive processes reflected in anticipatory adjustments to actions (motor and non-motor). Both recognize the importance of generating dynamically stable solutions. The recent progress in studies of the neural control of movements has led to a theory of hierarchical control with spatial referent coordinates for the effectors. This theory, in combination with the uncontrolled manifold hypothesis, allows quantifying the stability of actions with respect to salient variables. This approach has been used in the analysis of motor learning, changes in movements with typical and atypical development and with aging, and impaired actions by patients with various neurological disorders. It has been developed to address issues of kinesthetic perception. There seems to be hope that the two counter-directional approaches will meet and result in a single theoretical scheme encompassing biological phenomena from figuring out the best next move in a chess position to activating motor units appropriate for implementing that move on the chessboard.

Multi-modal Framework for Fetal Heart Rate Estimation: Fusion of Low-SNR ECG and Inertial Sensors.

This study presents a novel multi-modal framework for fetal heart rate extraction, which incorporates wearable seismo-cardiography (SCG), gyro-cardiography (GCG), and electrocardiography (ECG) readings from ten pregnant women. Firstly, a signal refinement method based on empirical mode decomposition (EMD) is proposed to extract the desired signal components associated with fetal heart rate (FHR). Afterwards, two techniques are developed to fuse the information from different modalities. The first method, named early fusion, is intended to combine the refined signals of different modalities through intra-modality fusion, intermodality fusion, and FHR estimation. The other fusion approach, i.e., late fusion, includes FHR estimation and intermodality FHR fusion. FHR values are estimated and compared with readings from a simultaneously-recorded cardiotocography (CTG) sensor. It is demonstrated that the best performance belongs to the late-fusion approach with 87.00% of positive percent agreement (PPA), 6.30% of absolute percent error (APE), and 10.55 beats-per-minute (BPM) of root-meansquare-error (RMSE).Clinical Relevance- The proposed framework allows for the continuous monitoring of the health status of the fetus in expectant women. The approach is accurate and cost-effective due to the use of advanced signal processing techniques and lowcost wearable sensors, respectively.

Neuron-specific spinal cord translatomes reveal a neuropeptide code for mouse dorsal horn excitatory neurons

The spinal dorsal horn harbors a sophisticated and heterogeneous network of excitatory and inhibitory neurons that process peripheral signals encoding different sensory modalities. Although it has long been recognized that this network is crucial both for the separation and the integration of sensory signals of different modalities, a systematic unbiased approach to the use of specific neuromodulatory systems is still missing. Here, we have used the translating ribosome affinity purification (TRAP) technique to map the translatomes of excitatory glutamatergic (vGluT2+) and inhibitory GABA and/or glycinergic (vGAT+ or Gad67+) neurons of the mouse spinal cord. Our analyses demonstrate that inhibitory and excitatory neurons are not only set apart, as expected, by the expression of genes related to the production, release or re-uptake of their principal neurotransmitters and by genes encoding for transcription factors, but also by a differential engagement of neuromodulator, especially neuropeptide, signaling pathways. Subsequent multiplex in situ hybridization revealed eleven neuropeptide genes that are strongly enriched in excitatory dorsal horn neurons and display largely non-overlapping expression patterns closely adhering to the laminar and presumably also functional organization of the spinal cord grey matter.

Multiple Sensory Modalities in Diurnal Geckos Is Associated with the Signaling Environment and Evolutionary Constraints.

SynopsisTo be effective, animal signals need to be detectable in the environment, but their development and expression require resources. For multimodal communication, investment in elaborating traits in one modality could reduce the elaboration of traits in other modalities. In Cnemaspis geckos, chemical signals for conspecific communication pre-dated the evolution of visual signals, allowing us to examine the potential trade-off in signal elaboration and the current habitat associations with signal use. We studied five species of Cnemaspis and quantified visual (patch size, color characteristics) and chemical (secretory composition) traits in males, as well as key environmental parameters (temperature, humidity, light) in each of their habitats. Within species, we found some trade-off in the elaboration of signals, as the strength of several components in the visual and chemical modalities were negatively associated. Strength of some signal components in each modality was also independently associated with specific environmental parameters that affect their detection (visual traits) and persistence (chemical traits). Specifically, species with larger, brighter, and more saturated color patches were found in habitats where the brightness and chroma of light were lower. Furthermore, environments with higher substrate temperature and higher relative humidity harbored species that produced secretions with a higher percentage of saturated and aromatic compounds. Thus, the elaboration of multimodal signals in this group of Cnemaspis geckos seems to increase the efficiency of communication in the signaling-environment, but the strength of signals in different modalities is constrained by trade-offs in signal expression.

Comparing perception of signals in different modalities during the cycling task: A field study

Cyclists are considered to be amongst the most vulnerable road users and the number of cyclists involved in crashes is increasing. One possibility to improve bicycle safety is the implementation of assistance systems, for instance by providing the information needed to avoid critical situations. However, it is not known how and what kind of signals can reliably be transmitted to cyclists, in particular as warnings. This study has the objective to investigate which signal type dependent of the modality and the route type can be perceived during the cycling task. Therefore, we conducted a semi-naturalistic cycling study with 56 participants where a 10 km long, pre-defined route was individually cycled while 36 signals (visual, auditory and vibro-tactile) were transmitted. The participants signalled the perception of a signal by pressing a button. Response rates differed significantly between signal modalities. While auditory signals performed best closely followed by vibro-tactile signals, visual signals were frequently missed. The route type had an effect on the perception of the signals. The influence of the route segments with haptic interference was not expected to be this large on the perception of vibro-tactile signals. The obtained results indicate how and in which situations the different modalities are suited to transmit information to cyclists.

The Sensory Machinery of the Head Louse Pediculus humanus capitis: From the Antennae to the Brain.

Insect antennae are sophisticated sensory organs, usually covered with sensory structures responsible for the detection of relevant signals of different modalities coming from the environment. Despite the relevance of the head louse Pediculus humanus capitis as a human parasite, the role of its antennal sensory system in the highly dependent relation established with their hosts has been barely studied. In this work, we present a functional description of the antennae of these hematophagous insects by applying different approaches, including scanning electron microscopy (SEM), anterograde antennal fluorescent backfills, and behavioral experiments with intact or differentially antennectomized lice. Results constitute a first approach to identify and describe the head louse antennal sensilla and to determine the role of the antenna in host recognition. SEM images allowed us to identify a total of 35–40 sensilla belonging to seven different morphological types that according to their external architecture are candidates to bear mechano-, thermo-, hygro-, or chemo-receptor functions. The anterograde backfills revealed a direct neural pathway to the ipsilateral antennal lobe, which includes 8–10 glomerular-like diffuse structures. In the two-choice behavioral experiments, intact lice chose scalp chemicals and warm surfaces (i.e., 32°C) and avoided wet substrates. Behavioral preferences disappeared after ablation of the different flagellomeres of their antenna, allowing us to discuss about the location and function of the different identified sensilla. This is the first study that integrates morphological and behavioral aspects of the sensory machinery of head lice involved in host perception.

Preference in female laboratory mice is influenced by social experience

Mice must perceive and interpret the signals they are presented with to evaluate potential mates, and should show a preference for high quality conspecifics. The present experiment was designed to investigate preference for acoustic, olfactory, and acoustic + olfactory signals in laboratory mice. Eighteen female CBA/CaJ mice, socially housed or isolated, were tested to evaluate preference for signals produced by males. Social experience influenced female preference for male USV playback and soiled bedding. Female mice who had experience with males showed a stronger preference for olfactory signals, whereas female mice with experience only with other females preferred USV playback. Isolated mice showed a greater preference for signals of different modalities presented together than socially experienced mice. This preference may have occurred because of increased motivation to make social contact. The findings of this experiment illustrate that social experience could play a role in mate choice.

Do sexual calls in female black capuchin monkeys (Sapajus nigritus) vary with fertility? An acoustic analysis.

Females across a range of animal taxa produce vocalizations and signals uniquely associated with periods of mating. While such signals may ultimately function to increase female attractivity to males, conflicting findings challenge the extent to which these signals co-vary in accordance with the probability of conception. Female black capuchin monkeys (Sapajus nigritus) display an elaborate repertoire of both vocal and visual components as part of their socio-sexual behavior, and previous analyses have shown that the rates of production of visual, but not vocal, components provide graded information on female ovulation. It remains possible, however, that the acoustic parameters of these sexual calls, rather than their rate of productions, co-vary with female fertility. To test this, we analyzed structural and temporal call parameters from estrous calls and post-copulatory calls recorded over five consecutive mating seasons in 12 sexually mature females at Iguazú National Park, Argentina. Calls given during the fertile phase of the female ovarian cycle were compared with those given during the non-fertile phase, as determined by profiles of female reproductive hormones. Similarly, within the fertile phase, we tested whether temporal or spectral acoustic parameters of calls gradually change with the approach of ovulation. We did not find any significant relationship between call parameters and the two measures of female fertility in either female estrous calls or post-copulatory calls. However, some differences between pre- and post-copulatory calls were apparent. Overall, our results indicate that sexual calls in black capuchin females do not provide precise information about the timing of ovulation, but may allow listeners to make probabilistic inferences about whether copulations have taken place. This, combined with previous findings, suggests that females in our study may use signals in different modalities to convey information about their fertility and sexual behavior with varying degrees of precision.

Intentional communication between wild bonnet macaques and humans

Comparative studies of nonhuman communication systems could provide insights into the origins and evolution of a distinct dimension of human language: intentionality. Recent studies have provided evidence for intentional communication in different species but generally in captive settings. We report here a novel behaviour of food requesting from humans displayed by wild bonnet macaques Macaca radiata, an Old World cercopithecine primate, in the Bandipur National Park of southern India. Using both natural observations and field experiments, we examined four different behavioural components—coo-calls, hand-extension gesture, orientation, and monitoring behaviour—of food requesting for their conformity with the established criteria of intentional communication. Our results suggest that food requesting by bonnet macaques is potentially an intentionally produced behavioural strategy as all the food requesting behaviours except coo-calls qualify the criteria for intentionality. We comment on plausible hypotheses for the origin and spread of this novel behavioural strategy in the study macaque population and speculate that the cognitive precursors for language production may be manifest in the usage of combination of signals of different modalities in communication, which could have emerged in simians earlier than in the anthropoid apes.

Directional vibration sensing in the leafcutter ant Atta sexdens.

ABSTRACTLeafcutter ants communicate with the substrate-borne component of the vibratory emission produced by stridulation. Stridulatory signals in the genus Atta have been described in different behavioural contexts, such as foraging, alarm signalling and collective nest building. Stridulatory vibrations are employed to recruit nestmates, which can localize the source of vibration, but there is little information about the underlying mechanisms. Our experiments reveal that time-of-arrival delays of the vibrational signals are used for tropotactic orientation in Atta sexdens. The detected time delays are in the same range as the time delays detected by termites. Chemical communication is also of great importance in foraging organization, and signals of different modalities may be combined in promoting the organization of collective foraging. Here we show that the tropotactic orientation to vibrational signals interacts with chemical communication signals.

Polarization multiplexing in large-mode-area waveguides and its application to signal enhancement in multiphoton microscopy

Soliton self-frequency shift (SSFS) is an efficient technique of generating broadband tunable femtosecond optical pulses. By using large-mode-area (LMA) waveguides such as photonic-crystal (PC) rods or LMA fibers, SSFS is capable of generating solitons with tens of or even >100 nJ pulse energy, enabling deep-tissue multiphoton microscopy (MPM) with the unprecedented imaging depth. MPM signals are proportional to the repetition rate of the laser. Here, we demonstrate an efficient technique of enhancing MPM signals in LMA waveguides, through polarization multiplexing, in both a PC rod with no polarization-maintaining (PM) structure and a PM LMA fiber. The collinear output soliton pulses with orthogonal linear polarizations show similar pulse energy, pulse width, and spectrum. We also demonstrate the application of this polarization multiplexing technique to MPM signal enhancement in biological tissues. Compared with single-polarization soliton excitation, excitation with polarization-multiplexed solitons can efficiently boost MPM signals in different modalities of MPM.

Production of and responses to unimodal and multimodal signals in wild chimpanzees, Pan troglodytes schweinfurthii

Animals communicate using a vast array of different signals in different modalities. For chimpanzees, vocalizations, gestures and facial expressions are all important forms of communication, yet these signals have rarely been studied together holistically. The current study aimed to provide the first comprehensive repertoire of flexibly combined (‘free’) multimodal (MM) signals, and assess individual and contextual factors influencing production of, and responses to, unimodal (UM) and MM signals in wild chimpanzees. In total, 48 different free MM signals were produced. MM signals were produced at a significantly lower rate than UM signals, but 22 of 26 focal animals were observed to produce free MM signals. The relative production rates of different types of UM and MM signals differed significantly between the behavioural contexts investigated, showing flexible use of signals across contexts. In contrast, individual factors such as age, sex or rank of signaller did not appear to influence the type of signal produced or the likelihood of eliciting a response. Finally, we compared recipient responses to free MM grunt-gesture signals and matched UM component signals and found that these MM signals were more likely to elicit a response than a grunt alone, but were as likely to elicit a response as the gesture alone. The overall findings point to a widespread capacity for wild chimpanzees to flexibly combine signals from different modalities and highlight the importance of adopting a multimodal approach to studying communication.

Two is company, three is a crowd

In many data-rich and safety-critical environments, multimodal displays (i.e., displays that present information in the visual, auditory, and tactile channels) are employed to support operators in dividing their attention across numerous tasks and sources of information. However, limitations of this approach are not well understood. Specifically, most research on the effectiveness of multimodal interfaces has examined the processing of only two concurrent signals in different modalities, mainly in vision and hearing. Anecdotal evidence and one empirical study (Hecht & Reiner, 2009) suggest that a person will likely fail to notice one or more signals when presented simultaneously with three or more unrelated multimodal stimuli. Therefore, the goals of this study were to (1) determine the extent to which people can notice and process three unrelated concurrent signals in different sensory channels and (2) whether and how this ability is modulated by age. Adults aged 65 years and older were of particular interest because individuals in this category represent the fastest growing segment of the U.S. population (U.S. Census Bureau, 2008), are known to suffer from various declines in sensory abilities (e.g, Li & Lindenberger, 2002; Stuart-Hamilton, 2012), and experience difficulties with divided attention (e.g., McDowd, Vercruyssen & Birren, 1991; Somberg & Salthouse, 1982) in general. Twelve younger (mean age: 23 years) and twelve older (mean age: 68 years) adults were presented with a series of singles, pairs, and triplets of visual, auditory, and tactile stimuli and asked to verbally indicate the modality of cue(s) they detected. The duration of each signal combination was one second. Prior to the task, crossmodal matching was performed to ensure that, subjectively, the stimuli were of equal intensity. Overall, the older adult group error rate was higher than that of the younger group (i.e., 3.3% and 0.98%, respectively). In particular, older adults often failed to notice the tactile cue when all three cues were combined. They also sometimes falsely reported the presence of a visual cue when presented with a combination of auditory and tactile cues. The findings from this work will be discussed in terms of underlying sensory and perceptual mechanisms. They will help inform the design of multimodal displays and ensure their usefulness across different age groups.

Tropical wrens rely more on acoustic signals than visual signals for inter- and intraspecific discrimination

Animals may use multiple signalling modalities to discriminate between conspecific versus heterospecific animals, or between individuals that represent a threat versus a mating opportunity. Multimodal signals used in intra- and interspecific discrimination can serve as redundant signals, or each modality may convey unique information. Furthermore, signals in different modalities may show different transmission properties through different habitats. In this study we investigated how two congeneric wrens, rufous-and-white wrens, Thryophilus rufalbus, and banded wrens, Thryophilus pleurostictus, use acoustic and visual signals for species discrimination in tropical forest habitats. We coupled song playback experiments with visual models to assess the importance of these signals, both in combination and in isolation. We assessed vegetation density in the territories of both species to assess whether more densely vegetated territories influence the use of visual signals. We presented both rufous-and-white wrens and banded wrens with conspecific and congeneric song treatments, model treatments and song-accompanied-by-model treatments. We found that both species responded strongly to song and song-accompanied-by-model treatments, but showed little or no response when the model was presented alone. These results suggest that wrens rely heavily on acoustic signals and very little on visual signals for discrimination. The species differed in their response to conspecific and congeneric trials, with rufous-and-white wrens showing little response to the congeneric trials but banded wrens responding strongly to both conspecific and congeneric trials. The asymmetrical response to the playback trials suggests that there may be a social dominance relationship between these two species, with rufous-and-white wrens being dominant over banded wrens. No previous studies have investigated the relative importance of acoustic and visual signals in males and females for species discrimination. Our results suggest that acoustic signals are more important than visual signals for inconspicuous animals living in dense environments.

The Basolateral Nucleus in the System of Reproductive Centers in the Amygdaloid Body of the Brain

This review systematizes data in the contemporary literature characterizing the basolateral nucleus of the amygdaloid body (AB) of the brain as a zone of sexual dimorphism (ZSD) and its interactions with other reproductive centers of the AB. The basolateral nucleus, like structures of the phylogenetically newer part of the AB, receives most of the pheromonal and olfactory information via the phylogenetically ancient corticomedial segment of the AB. Integrating it with sensory signals of different modalities arriving from the thalamus and cortical formations, the basolateral nucleus, along with the ZSD of the corticomedial group of structures, is involved in organizing sexual behavior and controlling gonadotropin secretion. Sexual dimorphism of the basolateral nucleus, which is involved in the functional system forming the emotions and memory mechanisms, determines the gender-dependent characteristics of their behavioral manifestations, especially in the body’s stress reactions.

The role of signals of different modalities in initiating vibratory communication in Nezara viridula

Abstract Signals of different modalities are involved in the behaviour of the green stink bug, Nezara viridula (L.) (Pentatomidae, Heteroptera). Long range attraction is mediated by male pheromones, resulting in aggregation of bugs on the same plant where vibratory signals, vision and various chemical signals become important. Both males and females sing spontaneously. When both are on the plant, males start vibratory communication as often as females. Females induce the exchange of vibratory signals spontaneously or triggered by the male pheromone while males initiate the duet either spontaneously or after seeing the female. Males and females sing spontaneously and respond to signals of different modalities more often in the daylight than in the dark. Long lasting autonomous emission of the female calling song is present when triggered by the male pheromone and males respond to female calling predominantly by the emission of the courtship song.

Duration perception in crossmodally-defined intervals

How humans perform duration judgments with multisensory stimuli is an ongoing debate. Here, we investigated how sub-second duration judgments are achieved by asking participants to compare the duration of a continuous sound to the duration of an empty interval in which onset and offset were marked by signals of different modalities using all combinations of visual, auditory and tactile stimuli. The pattern of perceived durations across five stimulus durations (ranging from 100ms to 900ms) follows the Vierordt Law. Furthermore, intervals with a sound as onset (audio-visual, audio-tactile) are perceived longer than intervals with a sound as offset. No modality ordering effect is found for visualtactile intervals. To infer whether a single modality-independent or multiple modality-dependent time-keeping mechanisms exist we tested whether perceived duration follows a summative or a multiplicative distortion pattern by fitting a model to all modality combinations and durations. The results confirm that perceived duration depends on sensory latency (summative distortion). Instead, we did not find evidence for multiplicative distortions. The results of the model and the behavioural data support the concept of a single time-keeping mechanism that allows for judgments of durations marked by multisensory stimuli.

Multimodal signals increase active space of communication by wolf spiders in a complex litter environment

Multimodal signals may compensate for environmental constraints on communication, as signals in different modalities vary in efficacy. We examined the influence of complex microhabitats on transmission of vibratory and visual signals of courting male Schizocosa ocreata wolf spiders (Araneae: Lycosidae) with laser Doppler vibrometry (LDV) and behavioral observations in lab and field. We measured maximum potential detection distance of visual and vibratory signals by females in laboratory mesocosms, recorded vibration signal attenuation on different substrates, and estimated transmission distances for male vibration signals in the field. We also determined effective line-of-sight visual detection distances in the field with laser distance measures. Together, these data were used to estimate the potential and effective active space of multimodal signals. LDV measures show leaves are highly conductive substrates for wolf spider vibratory signals compared to others (soil, wood, rock). For both visual and vibratory modes, lab estimates of maximum potential distance for signal transmission and detection (behavior studies) exceeded estimates of effective active space (signal attenuation, “vanishing point,” and “line-of-sight” measures). Field estimates of transmission distance for signal modes overlap, such that in close range (<20 cm), vibratory signals are more likely to be detected, while farther away, visual signals are more likely to be seen. These findings thus support current hypotheses regarding how multimodal communication might extend the range of overall signal active space or compensate for environmental constraints.